Activation Function Of Hidden Neurons Research Articles

Artifcial intelligence capabilities in evaluating effectiveness of non-medicinal treatment of obesity in children

Introduction. Non-drug therapy for obesity cannot always guarantee a positive result, which forces doctors and scientists from all over the world to look for new methods for analyzing the effectiveness of treatment, including using artifcial intelligence. Its active implementation can significantly improve the quality of diagnosis and prognosis of the disease. Purpose of the study. To evaluate the possibilities of using the artifcial intelligence system in predicting the effectiveness of non-drug therapy for obesity in children.Materials and methods. An artifcial neural network was built using the Statistica Neural Networks software package based on data from patients who were hospitalized at the Voronezh Children's Clinical Hospital of the VSMU n.a. N.N. Burdenko.Results. The study group included 60 children (30 boys and 30 girls), aged 8 to 16 years. We selected the parameters that, in our opinion, have the most signifcant impact on the effect of non-drug treatment of obesity: the presence and frequency of inpatient treatment; obesity complications; compliance with the regime of physical activity and dietary recommendations; dynamics of body weight during non-drug treatment. After training, the neural network MLP 5-5-1 was selected with determination coeffcients of 0.925231; 0.981940; 0.936712 for training, test and control samples. The learning error is 0.105782, the learning algorithm is BFGS. The activation function of hidden neurons is hyperbolic, and the output function is identical.Conclusion. The results of the study show that an artifcial neural network can be used to evaluate the effectiveness of non-drug treatment with a minimum error.

Design deep neural network architecture using a genetic algorithm for estimation of pile bearing capacity.

Determination of pile bearing capacity is essential in pile foundation design. This study focused on the use of evolutionary algorithms to optimize Deep Learning Neural Network (DLNN) algorithm to predict the bearing capacity of driven pile. For this purpose, a Genetic Algorithm (GA) was developed to select the most significant features in the raw dataset. After that, a GA-DLNN hybrid model was developed to select optimal parameters for the DLNN model, including: network algorithm, activation function for hidden neurons, number of hidden layers, and the number of neurons in each hidden layer. A database containing 472 driven pile static load test reports was used. The dataset was divided into three parts, namely the training set (60%), validation (20%) and testing set (20%) for the construction, validation and testing phases of the proposed model, respectively. Various quality assessment criteria, namely the coefficient of determination (R2), Index of Agreement (IA), mean absolute error (MAE) and root mean squared error (RMSE), were used to evaluate the performance of the machine learning (ML) algorithms. The GA-DLNN hybrid model was shown to exhibit the ability to find the most optimal set of parameters for the prediction process.The results showed that the performance of the hybrid model using only the most critical features gave the highest accuracy, compared with those obtained by the hybrid model using all input variables.

Two fast and accurate heuristic RBF learning rules for data classification

This paper presents new Radial Basis Function (RBF) learning methods for classification problems. The proposed methods use some heuristics to determine the spreads, the centers and the number of hidden neurons of network in such a way that the higher efficiency is achieved by fewer numbers of neurons, while the learning algorithm remains fast and simple. To retain network size limited, neurons are added to network recursively until termination condition is met. Each neuron covers some of train data. The termination condition is to cover all training data or to reach the maximum number of neurons. In each step, the center and spread of the new neuron are selected based on maximization of its coverage. Maximization of coverage of the neurons leads to a network with fewer neurons and indeed lower VC dimension and better generalization property. Using power exponential distribution function as the activation function of hidden neurons, and in the light of new learning approaches, it is proved that all data became linearly separable in the space of hidden layer outputs which implies that there exist linear output layer weights with zero training error. The proposed methods are applied to some well-known datasets and the simulation results, compared with SVM and some other leading RBF learning methods, show their satisfactory and comparable performance.

Symmetric extreme learning machine

Extreme learning machine (ELM) can be considered as a black-box modeling approach that seeks a model representation extracted from the training data. In this paper, a modified ELM algorithm, called symmetric ELM (S-ELM), is proposed by incorporating a priori information of symmetry. S-ELM is realized by transforming the original activation function of hidden neurons into a symmetric one with respect to the input variables of the samples. In theory, S-ELM can approximate N arbitrary distinct samples with zero error. Simulation results show that, in the applications where there exists the prior knowledge of symmetry, S-ELM can obtain better generalization performance, faster learning speed, and more compact network architecture.

A new approach for epileptic seizure detection using adaptive neural network

This paper presents new neural network models with adaptive activation function (NNAAF) to detect epileptic seizure. Our NNAAF models included three types named as NNAAF-1, NNAAF-2 and NNAAF-3. The activation function of hidden neuron in the model of NNAAF-1 is sigmoid function with free parameters. In the second model, NNAAF-2, activation function of hidden neuron is sum of sigmoid function with free parameters and sinusoidal function with free parameters. In the third model, NNAAF-3, hidden neurons' activation function is Morlet Wavelet function with free parameters. In addition, we implemented traditional multilayer perceptron (MLP) neural network (NN) model with fixed sigmoid activation function in the hidden layer to compare NNAAF models. The proposed models were trained and tested using 5-fold cross-validation to prove robustness of these models and to find the best model. We achieved 100% average sensitivity, average specificity, and approximately 100% average classification rate in all the models. It was seen that their speeds and the number of maximum iteration were changed for each model. The training time and the number of maximum iteration were reduced on about 50% using NNAAF-3 model. Hence it can be remarkable that NNAAF-3 is more suitable than the other models for real-time application.

A max-piecewise-linear neural network for function approximation

This paper proposes a Max-Piecewise-Linear (MPWL) Neural Network for function approximation. The MPWL network consists of a single hidden layer and employs the Piecewise-Linear (PWL) Basis Functions as the activation functions of hidden neurons. Since a PWL Basis Function possesses a simple functional form and universal representation capability, the MPWL network achieves a good balance between the computational simplicity and approximation accuracy. In addition, a PWL version of Back-Propagation (PBP) algorithm is developed, whose computational complexity is lower than the training algorithms for the Canonical PWL network, and the Back-Propagation algorithm for the sigmoid network with same number of training cycles. Another advantage of the MPWL network is its amenability to hardware implementation. This facilitates many applications such as nonlinear circuit synthesis, dynamic identification and control.